{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f375e8dd-3dde-46da-82bd-6e95e6cac08f",
   "metadata": {},
   "source": [
    "# Filtering out elements based on hard criteria\n",
    "\n",
    "In a lot of cases hard filtering is needed, when we specifically do not want the result set to contain some items, no matter how deep we scroll into the results. This can be achieved via the `.filter` clause in the `Query`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "beec4358-6221-4c62-89a5-7c381c8906db",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install superlinked==37.5.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e91ab315-38f2-42b9-80bf-c6dee1869ed1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from superlinked import framework as sl\n",
    "\n",
    "pd.set_option(\"display.max_colwidth\", 100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "ca8bbd4a-8a7e-49fc-8a2c-b322a008cac1",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Paragraph(sl.Schema):\n",
    "    id: sl.IdField\n",
    "    body: sl.String\n",
    "    author: sl.String\n",
    "    length: sl.Integer\n",
    "    tags: sl.StringList\n",
    "    is_active: sl.Boolean\n",
    "\n",
    "\n",
    "paragraph = Paragraph()\n",
    "\n",
    "body_space = sl.TextSimilaritySpace(text=paragraph.body, model=\"sentence-transformers/all-mpnet-base-v2\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "34529f84-e9b4-4b9e-9635-be3876f316e3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:18 superlinked.framework.dsl.index.index INFO   initialized index\n"
     ]
    }
   ],
   "source": [
    "paragraph_index = sl.Index(\n",
    "    [body_space],\n",
    "    fields=[paragraph.author, paragraph.body, paragraph.length, paragraph.tags, paragraph.is_active],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3839ad25-3c44-4a7a-a2a4-5ee9b321f5ee",
   "metadata": {},
   "source": [
    "> **_NOTE:_** The index definition requires the fields that we plan to create filters for."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34e39c98-7a1b-4a8a-8973-13d3eb7fb566",
   "metadata": {},
   "source": [
    "Now let's add some data and try it out!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "647d35a8-ea92-494c-ba2b-7472341f850b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:18 superlinked.framework.query.query_dag_evaluator INFO   initialized query dag\n",
      "17:27:18 superlinked.framework.online.online_dag_evaluator INFO   initialized entity dag\n",
      "17:27:18 superlinked.framework.dsl.executor.interactive.interactive_executor INFO   started in-memory app\n"
     ]
    }
   ],
   "source": [
    "source: sl.InMemorySource = sl.InMemorySource(paragraph)\n",
    "executor = sl.InMemoryExecutor(sources=[source], indices=[paragraph_index])\n",
    "app = executor.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "9424d137-d416-4faf-8e05-7984697d7ccf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:18 sentence_transformers.SentenceTransformer INFO   Load pretrained SentenceTransformer: sentence-transformers/all-mpnet-base-v2\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fac82bbeeb4941689b317423e9706344",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.online.online_dag_evaluator INFO   evaluated entities\n",
      "17:27:19 superlinked.framework.online.source.online_data_processor INFO   stored input data\n"
     ]
    }
   ],
   "source": [
    "source.put(\n",
    "    [\n",
    "        {\n",
    "            \"id\": \"paragraph-1\",\n",
    "            \"body\": \"The first thing Adam wrote.\",\n",
    "            \"author\": \"Adam\",\n",
    "            \"length\": 300,\n",
    "            \"tags\": [\"old\", \"interesting\"],\n",
    "            \"is_active\": False,\n",
    "        },\n",
    "        {\n",
    "            \"id\": \"paragraph-2\",\n",
    "            \"body\": \"The first thing Bob wrote.\",\n",
    "            \"author\": \"Bob\",\n",
    "            \"length\": 500,\n",
    "            \"tags\": [\"fresh\", \"dull\", \"useful\"],\n",
    "            \"is_active\": True,\n",
    "        },\n",
    "        {\n",
    "            \"id\": \"paragraph-3\",\n",
    "            \"body\": \"The second thing Adam wrote.\",\n",
    "            \"author\": \"Adam\",\n",
    "            \"length\": 400,\n",
    "            \"tags\": [\"interesting\", \"funny\", \"fresh\"],\n",
    "            \"is_active\": True,\n",
    "        },\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28139aac-84c2-41ab-8dbf-caccc1abd279",
   "metadata": {},
   "source": [
    "## Using the .filter clause\n",
    "\n",
    "### Comparisons\n",
    "Provides the opportunity to write filters on the result set. For example I can ask for articles written by Adam..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "22b38a70-ca5c-41ca-90ea-1330f0fa40b0",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>300</td>\n",
       "      <td>[old, interesting]</td>\n",
       "      <td>False</td>\n",
       "      <td>paragraph-1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>The second thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>400</td>\n",
       "      <td>[interesting, funny, fresh]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-3</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           body author  length                         tags  \\\n",
       "0   The first thing Adam wrote.   Adam     300           [old, interesting]   \n",
       "1  The second thing Adam wrote.   Adam     400  [interesting, funny, fresh]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0      False  paragraph-1               0.0  \n",
       "1       True  paragraph-3               0.0  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adam_query = sl.Query(paragraph_index).find(paragraph).filter(paragraph.author == \"Adam\").select_all()\n",
    "adam_result = app.query(adam_query)\n",
    "\n",
    "sl.PandasConverter.to_pandas(adam_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ddaac06-d59c-4076-a5c4-cda1434bf55f",
   "metadata": {},
   "source": [
    "> **_NOTE:_**  As we are only using filters, `similarity_score` is always 0, as there is effectively no vector search in this case."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45356bb2-fd9d-442b-880e-b2dce82c2d3b",
   "metadata": {},
   "source": [
    "...or not Adam."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "32a66530-3c8c-4cd5-a63a-d4f2191a3b7e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Bob wrote.</td>\n",
       "      <td>Bob</td>\n",
       "      <td>500</td>\n",
       "      <td>[fresh, dull, useful]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-2</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         body author  length                   tags  \\\n",
       "0  The first thing Bob wrote.    Bob     500  [fresh, dull, useful]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0       True  paragraph-2               0.0  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bob_query = sl.Query(paragraph_index).find(paragraph).filter(paragraph.author != \"Adam\").select_all()\n",
    "bob_result = app.query(bob_query)\n",
    "\n",
    "sl.PandasConverter.to_pandas(bob_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f886886-cf3b-4df5-b2ce-2c741d651170",
   "metadata": {},
   "source": [
    "I can also just simply filter for active paragraphs, using our boolean schema field."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "1c1cd0ac-3c19-409c-b2a4-38570bf0f4aa",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Bob wrote.</td>\n",
       "      <td>Bob</td>\n",
       "      <td>500</td>\n",
       "      <td>[fresh, dull, useful]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-2</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>The second thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>400</td>\n",
       "      <td>[interesting, funny, fresh]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-3</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           body author  length                         tags  \\\n",
       "0    The first thing Bob wrote.    Bob     500        [fresh, dull, useful]   \n",
       "1  The second thing Adam wrote.   Adam     400  [interesting, funny, fresh]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0       True  paragraph-2               0.0  \n",
       "1       True  paragraph-3               0.0  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "active_query = sl.Query(paragraph_index).find(paragraph).filter(paragraph.is_active.is_(True)).select_all()\n",
    "active_result = app.query(active_query)\n",
    "\n",
    "sl.PandasConverter.to_pandas(active_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efb55312-9a8c-4a0a-8b76-9c4581ac571b",
   "metadata": {},
   "source": [
    "Which can also be stated inversely."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "16257ae8-d575-4e31-9d1d-ac2bf3bebe44",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Bob wrote.</td>\n",
       "      <td>Bob</td>\n",
       "      <td>500</td>\n",
       "      <td>[fresh, dull, useful]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-2</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>The second thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>400</td>\n",
       "      <td>[interesting, funny, fresh]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-3</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           body author  length                         tags  \\\n",
       "0    The first thing Bob wrote.    Bob     500        [fresh, dull, useful]   \n",
       "1  The second thing Adam wrote.   Adam     400  [interesting, funny, fresh]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0       True  paragraph-2               0.0  \n",
       "1       True  paragraph-3               0.0  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "not_inactive_query = sl.Query(paragraph_index).find(paragraph).filter(paragraph.is_active.is_not_(False)).select_all()\n",
    "not_inactive_result = app.query(not_inactive_query)\n",
    "\n",
    "sl.PandasConverter.to_pandas(not_inactive_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "318cdacb-169c-4fa7-bf70-c4529496e99a",
   "metadata": {},
   "source": [
    "An alternative syntax is also possible, but that will not satisfy most linters, so an ignore is recommended in this case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "f3930fec-d229-4c78-a9f6-1be50c86c262",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Bob wrote.</td>\n",
       "      <td>Bob</td>\n",
       "      <td>500</td>\n",
       "      <td>[fresh, dull, useful]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-2</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>The second thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>400</td>\n",
       "      <td>[interesting, funny, fresh]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-3</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           body author  length                         tags  \\\n",
       "0    The first thing Bob wrote.    Bob     500        [fresh, dull, useful]   \n",
       "1  The second thing Adam wrote.   Adam     400  [interesting, funny, fresh]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0       True  paragraph-2               0.0  \n",
       "1       True  paragraph-3               0.0  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "active_query_alt_syntax = (\n",
    "    sl.Query(paragraph_index)\n",
    "    .find(paragraph)\n",
    "    .filter(paragraph.is_active == True)  # noqa: E712 # pylint: disable=singleton-comparison\n",
    "    .select_all()\n",
    ")\n",
    "active_result_alt_syntax = app.query(active_query_alt_syntax)\n",
    "\n",
    "sl.PandasConverter.to_pandas(active_result_alt_syntax)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6df517e-cff2-4251-9e92-a03615fd63d2",
   "metadata": {},
   "source": [
    "It is possible to filter for being greater or smaller (and equal) to certain field."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "06dbb4cf-b2be-44cd-8bd7-0e2e5006b61b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>300</td>\n",
       "      <td>[old, interesting]</td>\n",
       "      <td>False</td>\n",
       "      <td>paragraph-1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>The second thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>400</td>\n",
       "      <td>[interesting, funny, fresh]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-3</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           body author  length                         tags  \\\n",
       "0   The first thing Adam wrote.   Adam     300           [old, interesting]   \n",
       "1  The second thing Adam wrote.   Adam     400  [interesting, funny, fresh]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0      False  paragraph-1               0.0  \n",
       "1       True  paragraph-3               0.0  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "greater_equal_query = sl.Query(paragraph_index).find(paragraph).filter(paragraph.length <= 400).select_all()\n",
    "greater_equal_result = app.query(greater_equal_query)\n",
    "\n",
    "sl.PandasConverter.to_pandas(greater_equal_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af09eefd-cea4-4534-981b-3aa873c13e80",
   "metadata": {},
   "source": [
    "### Logical operations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b20533b7-5cce-47e3-aa0a-83cd08de878d",
   "metadata": {},
   "source": [
    "And we can also stack multiple filters to form AND type of relationship."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b0741b88-5996-44e9-a198-d528aa44edf9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>300</td>\n",
       "      <td>[old, interesting]</td>\n",
       "      <td>False</td>\n",
       "      <td>paragraph-1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          body author  length                tags  is_active  \\\n",
       "0  The first thing Adam wrote.   Adam     300  [old, interesting]      False   \n",
       "\n",
       "            id  similarity_score  \n",
       "0  paragraph-1               0.0  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stacked_query = (\n",
    "    sl.Query(paragraph_index)\n",
    "    .find(paragraph)\n",
    "    .filter(paragraph.author == \"Adam\")\n",
    "    .filter(paragraph.body == \"The first thing Adam wrote.\")\n",
    "    .select_all()\n",
    ")\n",
    "stacked_result = app.query(stacked_query)\n",
    "\n",
    "sl.PandasConverter.to_pandas(stacked_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c84afea",
   "metadata": {},
   "source": [
    "It is possible to filter for being greater or smaller (and equal) to certain field"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "207165bc-e5eb-4fa1-a529-29576b6e9874",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>300</td>\n",
       "      <td>[old, interesting]</td>\n",
       "      <td>False</td>\n",
       "      <td>paragraph-1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>The first thing Bob wrote.</td>\n",
       "      <td>Bob</td>\n",
       "      <td>500</td>\n",
       "      <td>[fresh, dull, useful]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-2</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          body author  length                   tags  \\\n",
       "0  The first thing Adam wrote.   Adam     300     [old, interesting]   \n",
       "1   The first thing Bob wrote.    Bob     500  [fresh, dull, useful]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0      False  paragraph-1               0.0  \n",
       "1       True  paragraph-2               0.0  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "or_query = (\n",
    "    sl.Query(paragraph_index)\n",
    "    .find(paragraph)\n",
    "    .filter((paragraph.author == \"Bob\").or_(paragraph.length == 300))\n",
    "    .select_all()\n",
    ")\n",
    "or_result = app.query(or_query)\n",
    "\n",
    "sl.PandasConverter.to_pandas(or_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47e62151-bdb6-4424-baeb-42815dd33852",
   "metadata": {},
   "source": [
    "... or simply using the well known ways of `__or__`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "5f2d4f4b-fe88-46a9-ad76-e0c0b1b36eb1",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Bob wrote.</td>\n",
       "      <td>Bob</td>\n",
       "      <td>500</td>\n",
       "      <td>[fresh, dull, useful]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-2</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>The second thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>400</td>\n",
       "      <td>[interesting, funny, fresh]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-3</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           body author  length                         tags  \\\n",
       "0    The first thing Bob wrote.    Bob     500        [fresh, dull, useful]   \n",
       "1  The second thing Adam wrote.   Adam     400  [interesting, funny, fresh]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0       True  paragraph-2               0.0  \n",
       "1       True  paragraph-3               0.0  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "or_query_ = (\n",
    "    sl.Query(paragraph_index)\n",
    "    .find(paragraph)\n",
    "    .filter((paragraph.author == \"Bob\") | (paragraph.length == 400))\n",
    "    .select_all()\n",
    ")\n",
    "or_result_ = app.query(or_query_)\n",
    "\n",
    "sl.PandasConverter.to_pandas(or_result_)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9578332-6e77-46eb-9034-e1b960e55c1b",
   "metadata": {},
   "source": [
    "### Set operations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0b379f2-0249-45db-b2ac-bd90e118c957",
   "metadata": {},
   "source": [
    "We can filter for a paragraph having an author from a group of possible authors using the `in_` operator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "4971d788-0042-46b4-b2f7-cd756d9bc2ba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>300</td>\n",
       "      <td>[old, interesting]</td>\n",
       "      <td>False</td>\n",
       "      <td>paragraph-1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>The second thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>400</td>\n",
       "      <td>[interesting, funny, fresh]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-3</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           body author  length                         tags  \\\n",
       "0   The first thing Adam wrote.   Adam     300           [old, interesting]   \n",
       "1  The second thing Adam wrote.   Adam     400  [interesting, funny, fresh]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0      False  paragraph-1               0.0  \n",
       "1       True  paragraph-3               0.0  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "in_query = sl.Query(paragraph_index).find(paragraph).filter(paragraph.author.in_(sl.Param(\"filter_list\"))).select_all()\n",
    "in_result = app.query(in_query, filter_list=[\"Alice\", \"Adam\", \"Amon\"])\n",
    "\n",
    "sl.PandasConverter.to_pandas(in_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ded4d3e4-63d3-4bf0-b8c7-78beea4db922",
   "metadata": {},
   "source": [
    "Or for a paragraph has an author that is not part of a list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "7ad370b2-a795-4d4b-812c-4be84214cb08",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Bob wrote.</td>\n",
       "      <td>Bob</td>\n",
       "      <td>500</td>\n",
       "      <td>[fresh, dull, useful]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-2</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         body author  length                   tags  \\\n",
       "0  The first thing Bob wrote.    Bob     500  [fresh, dull, useful]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0       True  paragraph-2               0.0  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "not_in_query = (\n",
    "    sl.Query(paragraph_index).find(paragraph).filter(paragraph.author.not_in_(sl.Param(\"filter_list\"))).select_all()\n",
    ")\n",
    "not_in_result = app.query(not_in_query, filter_list=[\"Alice\", \"Adam\", \"Amon\"])\n",
    "\n",
    "sl.PandasConverter.to_pandas(not_in_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f844ac43-ac36-4506-830e-300ba03466ac",
   "metadata": {},
   "source": [
    "List type attributes could be filtered conditioning on containing a value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "ce8db820-6d0a-45ac-b72c-d8ae690dce8a",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Bob wrote.</td>\n",
       "      <td>Bob</td>\n",
       "      <td>500</td>\n",
       "      <td>[fresh, dull, useful]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-2</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>The second thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>400</td>\n",
       "      <td>[interesting, funny, fresh]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-3</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           body author  length                         tags  \\\n",
       "0    The first thing Bob wrote.    Bob     500        [fresh, dull, useful]   \n",
       "1  The second thing Adam wrote.   Adam     400  [interesting, funny, fresh]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0       True  paragraph-2               0.0  \n",
       "1       True  paragraph-3               0.0  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "contains_query = (\n",
    "    sl.Query(paragraph_index).find(paragraph).filter(paragraph.tags.contains(sl.Param(\"filter_tag\"))).select_all()\n",
    ")\n",
    "contains_any_single_result = app.query(contains_query, filter_tag=[\"fresh\"])\n",
    "\n",
    "sl.PandasConverter.to_pandas(contains_any_single_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "560d22bb-a1e6-45c5-9a97-bed13dd501f4",
   "metadata": {},
   "source": [
    "Contains with a `list` input work as the industry practice, returns entities containing either of the listed elements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "9dcaa1f6-b6f4-4598-bfc4-97e8179610d2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>300</td>\n",
       "      <td>[old, interesting]</td>\n",
       "      <td>False</td>\n",
       "      <td>paragraph-1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>The first thing Bob wrote.</td>\n",
       "      <td>Bob</td>\n",
       "      <td>500</td>\n",
       "      <td>[fresh, dull, useful]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-2</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>The second thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>400</td>\n",
       "      <td>[interesting, funny, fresh]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-3</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           body author  length                         tags  \\\n",
       "0   The first thing Adam wrote.   Adam     300           [old, interesting]   \n",
       "1    The first thing Bob wrote.    Bob     500        [fresh, dull, useful]   \n",
       "2  The second thing Adam wrote.   Adam     400  [interesting, funny, fresh]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0      False  paragraph-1               0.0  \n",
       "1       True  paragraph-2               0.0  \n",
       "2       True  paragraph-3               0.0  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "contains_any_multiple_query = (\n",
    "    sl.Query(paragraph_index).find(paragraph).filter(paragraph.tags.contains(sl.Param(\"filter_tag\"))).select_all()\n",
    ")\n",
    "contains_any_multiple_result = app.query(contains_any_multiple_query, filter_tag=[\"fresh\", \"interesting\"])\n",
    "\n",
    "sl.PandasConverter.to_pandas(contains_any_multiple_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7dbb2899-5141-4b5f-9984-671acbb97788",
   "metadata": {},
   "source": [
    "To filter for entities containing all of the listed elements, stack filters..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "512c4cb5-19da-44c2-9939-b202488f3c78",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Bob wrote.</td>\n",
       "      <td>Bob</td>\n",
       "      <td>500</td>\n",
       "      <td>[fresh, dull, useful]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-2</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         body author  length                   tags  \\\n",
       "0  The first thing Bob wrote.    Bob     500  [fresh, dull, useful]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0       True  paragraph-2               0.0  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "contains_all_stacked_query = (\n",
    "    sl.Query(paragraph_index)\n",
    "    .find(paragraph)\n",
    "    .filter(paragraph.tags.contains(sl.Param(\"filter_tag_1\")))\n",
    "    .filter(paragraph.tags.contains(sl.Param(\"filter_tag_2\")))\n",
    "    .select_all()\n",
    ")\n",
    "contains_all_stacked_result = app.query(contains_all_stacked_query, filter_tag_1=[\"fresh\"], filter_tag_2=[\"useful\"])\n",
    "\n",
    "sl.PandasConverter.to_pandas(contains_all_stacked_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4eed5c3",
   "metadata": {},
   "source": [
    "... or use contains_all."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "cce6095c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Bob wrote.</td>\n",
       "      <td>Bob</td>\n",
       "      <td>500</td>\n",
       "      <td>[fresh, dull, useful]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-2</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         body author  length                   tags  \\\n",
       "0  The first thing Bob wrote.    Bob     500  [fresh, dull, useful]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0       True  paragraph-2               0.0  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "contains_all_query = (\n",
    "    sl.Query(paragraph_index).find(paragraph).filter(paragraph.tags.contains_all(sl.Param(\"filter_tag\"))).select_all()\n",
    ")\n",
    "contains_all_result = app.query(contains_all_query, filter_tag=[\"fresh\", \"useful\"])\n",
    "\n",
    "sl.PandasConverter.to_pandas(contains_all_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "101ee56a-bf17-4dd6-9864-c4ad37358db3",
   "metadata": {},
   "source": [
    "The same can be done using not_contains, too. With a single..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "43fcf41a-029a-4258-81e3-e62d930608be",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The first thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>300</td>\n",
       "      <td>[old, interesting]</td>\n",
       "      <td>False</td>\n",
       "      <td>paragraph-1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          body author  length                tags  is_active  \\\n",
       "0  The first thing Adam wrote.   Adam     300  [old, interesting]      False   \n",
       "\n",
       "            id  similarity_score  \n",
       "0  paragraph-1               0.0  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "not_contains_query = (\n",
    "    sl.Query(paragraph_index).find(paragraph).filter(paragraph.tags.not_contains(sl.Param(\"filter_tag\"))).select_all()\n",
    ")\n",
    "not_contains_result = app.query(not_contains_query, filter_tag=[\"fresh\"])\n",
    "\n",
    "sl.PandasConverter.to_pandas(not_contains_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1638a9b0-88c5-4881-9737-e890042830f5",
   "metadata": {},
   "source": [
    "... or with multiple values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "de78e986-a816-438c-be70-88414315c314",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17:27:19 superlinked.framework.query.query_dag_evaluator INFO   evaluated query\n",
      "17:27:19 superlinked.framework.dsl.executor.query.query_executor INFO   executed query\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>body</th>\n",
       "      <th>author</th>\n",
       "      <th>length</th>\n",
       "      <th>tags</th>\n",
       "      <th>is_active</th>\n",
       "      <th>id</th>\n",
       "      <th>similarity_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The second thing Adam wrote.</td>\n",
       "      <td>Adam</td>\n",
       "      <td>400</td>\n",
       "      <td>[interesting, funny, fresh]</td>\n",
       "      <td>True</td>\n",
       "      <td>paragraph-3</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           body author  length                         tags  \\\n",
       "0  The second thing Adam wrote.   Adam     400  [interesting, funny, fresh]   \n",
       "\n",
       "   is_active           id  similarity_score  \n",
       "0       True  paragraph-3               0.0  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "not_contains_multiple_query = (\n",
    "    sl.Query(paragraph_index).find(paragraph).filter(paragraph.tags.not_contains(sl.Param(\"filter_tag\"))).select_all()\n",
    ")\n",
    "not_contains_multiple_result = app.query(not_contains_multiple_query, filter_tag=[\"old\", \"dull\"])\n",
    "\n",
    "sl.PandasConverter.to_pandas(not_contains_multiple_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90d86265-369c-462a-be94-16fff6e4c515",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "We are supporting\n",
    "\n",
    "#### Comparison operations\n",
    "* the `==` and `!=` operators,\n",
    "* the `<`, `>`, `<=` and `>=` operators,\n",
    "#### Logical operations\n",
    "* creating `AND` relationships by stacking filters,\n",
    "* and using the `.or_` or simply `|` to create `OR` relationships.\n",
    "#### Set operations\n",
    "* the `.in_` and `.not_in_` operators test String fields having a value from a collection,\n",
    "* the `.contains` and `not_contains_` operators test StringList containing either of the values from a collection,\n",
    "* the `.contains_all` operator tests StringList containing all of the values from a collection."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "superlinked-py3.11",
   "language": "python",
   "name": "superlinked-py3.11"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
